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Introduction 

The goal of this paper is to review the aspects of cognitive 
science that appear to relate best to efforts to use 
electrical and magnetic recording to understand the function of 
brain systems. To meet this goal it is important to understand 
the changes that have taken place in recent years, both in our 
understanding of brain function and in our understanding of 
cognition. It is on the latter changes that I plan to 
concentrate in this paper. 

The term cognitive science relates to efforts by students of 
psychology, linguistics and artificial intelligence, among 
others, to produce a fundamental analysis of natural and 
artificial intelligence. This area is of enormous breadth and 
there would be no possibility of a thorough review here. Instead 
I first attempt to develop a framework which describes work in 
cognition at several levels of analysis most appropriate for the 
goal of developing a relation to underlying neural systems. I 
then then analyze recent work on spatial attention in more detail 
as a model use of the framework to guide integrative research. 
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F ranework 

Figure 1 provides an overall framework of five levels that 


Insert Figure 1 about here 

seem to me to provide a basis for establishing empirical 
relationships between complex cognitive activity and brain 
systems. In the history of research on localization of function 
from the time of phrenology to the present there has been 
greatly oversimplified conceptions of cognition. Phrenology 
sought to localize very general mental faculties or cognitive 
traits that might underly the ability to compose music, 
play chess or perform some other complex skill. 

Following the development of behaviorism, the study 
of cognition was often confined to what could be observed in 
overt behavior, or with the aid of a very few simple 
internal constructs, mostly motivational, that tied internal 
systems to overt behavior. This approach still dominates in the 
study of behavioral neuroscience. 

In the last twenty years, complex cognitive tasks 
such as playing chess, reading, or manipulating visual images, 
have been subjected to detailed analysis. In his study of 
imagery, for example, Kosslyn (1980) postulates twelve elementary 
mental operations such as picture, find, put, image, regenerate, 
look for, scan. For example, the scan function performs the 
operation of "moving all points in surface matrix along a vector; 
fills in new material at leading edge via an inverse mapping 
function." Each operation performs a specific cognitive 
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function. 

When these mental operations are concatenated in a flow 
diagram, one has a computational model in the sense that it 
should allow a computer to perform the prescribed imagery task. 
Kosslyn uses as an example the task of examining whether an 
image of an automobile does or do-s <.■ t contain a spare tire. 

In recent years some aspects of tasks such as reading 
(Rummelhart & McClelland, 1982) have been analyzed in terms of 
computational models. These models, show that the set of 
elementary operations proposed by the model are sufficient to 
produce the cognitive performance described. The top two levels 
of Figure 1 are at the level of cognitive science in that they 
deal with efforts to provide a sufficient basis for an 
electro-mechanical system to perform the type of cognition listed 
(e.g., chess playing or imagery). They do not necessarily tell 
us how a human mind, still less a human brain, performs these 
operations. Nonetheless, these computational models provide us 
with a way of analyzing cognition that shows very clearly that 
cognitive tasks may be viewed as consisting of elementary 
operations which are combined in complex programs to solve the 
overall task. 

In order to convert the abstract elementary operations 
of cognitive science to an analysis of human mental processes 
it is necessary to examine the components of these operations. 
Many such operations have been examined by chronometric 
experiments (Posner, 1978). These chronometric experiments 
require human subjects to perform elementary operations such 
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as "scanning a list”, "matching items", "zooming", postulated by 
the cognitive models. Such experimental studies have shown 
evidence of time locked component facilitations and inhibitions 
that occur in the process of performing these elementary 
operations. For example, suppose a person is required to scan a 
list of digits to determine if a single probe digit is a member 
of a previously presented list (Sternberg, 1966). The underlying 
processes could be examined by measuring the reaction time to 
respond “yes" or "no" as a function of number of items in the 
list. This analysis allows a detailed exrnination of a mental 
operator similar to one posited by many computer programs that 
require comparing a target with items stored in a list. If a 
human being is required to perform this task it takes 
approximately 30 milisec per digit as the length of the list is 
increased from 1 to 6 digits. 

The time locking of this putative comparison operation is 
impressive. Moreover, we know that the activation of any item 
during the comparison process produces a facilitation in 
processing items that are similar to it (Posner, 1978). For 
example, as the subject thinks about the digit 3 he potentiates 
the efficiency (e.g., speeds reaction time or reduces threshold) 
with which that visual digit is handled. In addition, we also 
know that when one attends actively to a digit, there will be an 
inhibition in the processing of items not sharing that pathway 
(Neely, 1977). Thus, the elementary operation involved in com¬ 
paring the probe digit with items in store may be studied in 
terms of time locked facilitations and inhibitions that affect 



probed responses. Some mental operations, including matching, 
naming, rotations, zooming, have been studied in terms of 
component facilitations and inhibitions. The results provide 
a psychological or information processing account of the 
underlying mental events involved in the task. These operations 
take place in real time and at a much slower rate than woula cf 
the case for existing computer sytems. 

Some psychologists and philosophers have stressed the 
sufficiency of this kind of component analysis in providing a 
basis for information processing models of cognition. They argue 
it is unneccessary to go further and ask to what extent are 
neural systems related to such components. However, other 
psychologists, including most readers of this journal, do wish to 
go further. 

Indeed there is reasonable evidence that components of the 
event related potential are systematically related to such 
component operations. For example, in 1978 I reviewed evidence 
that the time for release of the P-300 was related to the cegree 
of priming or activation of an underlying psychological pathway. 
Evidence since then (Ouncan-Johnson & Donchin, 1982) has shown 
that the latency and amplitude of the P-300 is systematically 
related to the benefits obtained in priming a pathway by a 
pre-cue. There is a good deal of independence between the degree 
of P-300 change and changes in reaction times with cuing, 
suggesting that the P-300 indexes somewhat different mental 
processes than are indexed by reaction time. 


Naatanen (1982) has shown that systematic negative shifts in 





the event related potential can be associated with the direction 
of the subject's attention. For example, if subjects are 
attending to the right ear, stimuli on that ear will show a 
negative shift with respect to stimuli occurring on an unattended 
channel (e.g. left ear) that takes place approximately 100 
nillisec after input. Maataner, argues that the latency of these 
negative shifts depend upon the extent of processing prior to 
reaching a level at which attention is directed. If attention is 
directed to a more complicated aspect of stimuli, e.g., 
frequency, the negative shift will occur later than if it is 
directed toward ear of entry. Harter and Guido (1980) have shown 
systematic negative shifts occurring about 200 msec for visual 
stimuli that match the spatial frequency to which the subject's 
attention has been directed. Since we know that attending to a 
particular pathway will activate that pathway (McLean & Shulnan, 
1973) it seems reasonable to suppose that the processing 
negativity discussed by Naatanen is, in fact, a brain sign 
related to the facilitation obtained in chronometric 
experiments. 

This assumption has a number of remaining difficulties. 

Most of the performance priming experiments use a trial by 
trial design. The prime is introduced at the start of the trial 
and a subsequent target on that trial is shown to be affected by 
the prime. However, most of the work reviewed by Naatanen 
requires successive presentation of a number of stimuli and 
indeed, the negative shift is reduced or lost when there is a 
long time between successive inputs. Most of the chronometric 




experiments seem to indicate that facilitation can occur either 
automatically from the presentation of the stimulus itself or 


from an act of attention. fiaatanen suggests, although his 
experiments do not prove, that attention is the necessary 
condition for the negative shifts found in the EEG . 

The very suggestive work from the EE3 can now be 
supplemented by other methods of relating component facilitations 
anc inhibitions to brain systems. These other methods include 
magnetic potentials,, study of blood flow, and the study 
the effects of brain lesions. For example, Knight, Hill *d , 
Woods & Neville (1980) have shown that lesions in the pa i< a 1 
lobe will affect the degree of processing negativity obtained 
when subjects are instructed to attend to one ear rather than the 
other. 

In the case of attention to visual locations there are also 
data available from single cell recording that do appear to 
converge upon our effort to relate component facilitations 
to underlying brain systems. During the last fifteen years 
neurophysiologists have been able to record from single cells in 
alert monkeys doing cognitive tasks. Summaries of this work 
(Wurtz, Goldberg l Robinson, 1980) indicate that single cells in 
the posterior parietal lobe of alert monkeys show a phenomenon 
called selective enhancement. Such cells show enhanced rates of 
firing when a target occurs in their receptive field and 
attention is directed toward that receptive field as compared 
to conditions when a target occurs in their receptive field and 



attention is directed elsewhere. Selective enhancement to snirts 
of covert attention (unaccompanied ly eye mo ven.e n t s ) dees not 
occur in ether areas of the brain such as frontal eye field, 
striate cortex, or superior colliculus. The convergence cf 
evidence from processing negativity studies using EES reccrcing 
and fror, the single cell results suggests that an understand r,g 
of the component processes involved in orienting attention to 
positions in space ray be a particularly good one for testing *ne 
overall framework outlined in Figure 1. 

Covert Orienting 

One cognitive operation that we have in comron with other 
animals is the ability to shift attention fror position to 
position in the visual field. This activity is cognitive ir, the 
sense that it requires an alert organism in active contact with 
its environment. Moreover, this operation of shifting attention 
is a very important component of a number of cognitive models. 

For example, Kosslyn's (1980) scan mechanism allows a shift of 
attention spatially from one part of the visual image to another. 
Similarly, models of the visual system (Feldman, 1982; Treisran 
£ Gelade, 1980) postulate an ability to bring attention to any 
location thus making available mental apparatus allowing more 
complex computations that can be performed in parallel across the 
visual field. This is often done by overt movements cf the head 
and eyes which align the fovea with the area of the visual field 
of interest. However, in recent years it has been shewn that it 
is possible to direct visual attention to a location within the 
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visual field without any ovri-t orientation or heaC a no e-es 
(?osner, 19oU) Such covert arientinc of attertion can Le r.ea sure", 
ty chances in efficiency, particularly in the latency anc 
threshold for detecting events that occur at tee location to 
which attention has Seen d i r oc tec in conearisen to other 
locations of similar eccentricity in the visual field. 

My colleagues anr ] (Posner, 1930; Posner & Cohen, 1964; 
Posner, Cohen l P a f a 1 , 1966; Posner, Walker £ Friedrich £ Ratal , 
in press) have studied a very simple task which erUodies many of 
the features of covert orienting. In this task, the subject's 
eyes are fixated at a central location on a cathode ray tube. At 
the beginning cf each trial, a cue is presentee by brightening 
a square located 7 degrees to the left or right cr rixation. The 
cue remains present for 150 msec. Following the onset of tne cue, 
a target stimulus is presented either at the cued location or on 
the opposite side of the CRT. The results of a typical experiment 
cf this type is shown in Figure 2. The X axis indicates the time 

Insert Figure 2 about here 

between onset of the cue and the target. Plotted cn the Y axis 
is the reaction tine to respond to a target. The top two lines 
(souare symbols) are from the first day in which the target was 
equally likely to occur on the cuea or uncuea side. Since this 
is the first day, subjects are a bit slower than on the second 
day. The results show clearly the rapid advantage in reaction 
time for the cued side (solid squares) over the uncued side 
(open squares) that begins about 5>9 after the cue and remains 






present over the entire interval except at the last point where 



there is a slight crossover. SuLjocts art systematically faster 
on the cue a sice than on the on cuee sice. The circles show 
conditions when the cuec sice has a probability .8 of getting a 
target. JnCer these conditions, the subjects are also faster on 
tne cued sice Isolid circles) tnan tne uncuec siae (open circles) 
ana reMain so throughout the interval. Even when the cue 
signifies that the target is unlikely (.2) to occur on the cuea 
side (triangles) there is a temporary facilitation on the siae of 
the cued (solid triangles) that is replaced after 300 mi 11isec by 
an advantage to the side which has the higher probability of the 
target (uncuec side, open triangles). 

What do these results indicate? First, the advantage of the 
cued siae over the uncued side arises because the subject's 
attention is drawn toward the cue. There is evidence in the 
literature supporting this explanation, since similar advantages 
are found when the cue is a central arrow, which incicates the 
side likely to get the target stimulus (Joniaes, 1981; Posner, 
1980). There does not need to be any peripheral energy change to 
obtain this result, although such an energy change is an 
efficient way of summoning the subject's attention. Figure 2 also 
illustrates this point because when the probability is .8 that 
the detection stimulus will occur on the uncued side, after 300 
millisec subjects show a strong advantage in reaction tine to 
targets on the uncued side. Indeed, there is also evidence that 
suggests that the reaction time advantages which we show for cuea 
targets are also accompanied by a reduction in the threshold for 
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detecting such targets (Bash in ski i. Bachrach, i980 ) . The use 
of central cues or probability differences show that the 
facilitation involves a mechanism that can either be pulled by a 
cue at the target location or directed from a cue that merely 
provides information about the target location. 

\ second point is the impressive time locking of the 
change in efficiency to pre sentation of cues. Time locking is 
very important if we are going to relate these results to neural 
systems. Within 500 mi 11isec we see first an advantage on the 
cued sine and then an advantage toward the high probability side 
whether cued or not. 

The change in attention t hi a t occurs in covert orienting 
can be seen to involve three more elementary mental operations. 
These are shown in Figure 3. They include disengagement from 

Insert Figure 3 about here 

the current focus of attention which may in this task be at 
fixation, movement across the visual field from the current 
location of attention to the target and finally, engagement in 
the target. The time to disengage attention from a current focus 
is a function of the depth of processing at that focus (LaBerge, 
1974). That is, subjects more heavily engaged in processing a 
stimulus at some target location will take longer to produce 
disengagement. There is evidence that attention movements are in 
some sense an analog operation. Kosslyn's model indicates that 
the scanning of a visual images is analog and his own data from 
experiments requiring subjects to access different locations in 







an image shows a linear relationship between RT and the distance 
from the current location of attention to the target location. 
Tsai (1983) has shown similar linear relationships between the 
distance of a visual target from fixation and the time following 
a cue when one obtains maximum facilitation at the target 
location. His data indicate that attention moves at the rate of 
about 8 millisec per aegree. This figure fits quite well with 
the data shown in Figure 2 in which a target about 7 degrees 
from location shows facilitation by about 50 millisec following 
the cue. More direct evidence of an analog movement across the 
visual field has been obtained by Shulman, Remington & McLean 
(1979) who showed faciliation of probe events at intermediate 
locations between fixation and the target occurs at times 
intermediate between introduction of the cue ana finding maximal 
facilitation at the target location. 

Once attention reaches the target location, it seems 
reasonable to postulate that it will take more time to engage 
the target task depending upon the complexity of the information 
processing necessary to reach the level of the system 
correspond! - ng to the task instructions given the subject. 

A final characteristic of the demonstration experiment 
illustrated in Figure 2 is the tendency for the advantage of the 
cued side over the uncued side to dissappear over time. We 
(Posner £ Cohen, 1904) believe that this is due to an inhibitory 
effect of the peripheral cue. This inhibition is seen most 
clearly if attention is summoned first to the periphery and then 
back to fixation. The previously cued location is now inhibited 
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in comparison to other locations. Thus, it is possible by using 
peripheral cues to study time locked facilitation followed by 
inhibition. Inhibition is striking in Figure 2 when one compares 
the dottea triangles with the solid circles at 500 millisec 
delay. Both these conditions represent locations at which a 
target will occur with .8 probability, but the latter is on the 
side that was formerly cued. Even though the cue has disappeared 
this side is slower than the comparable uncued condition. 

Posner and Cohen (1984) have proposed a hypothesis 
concerning the functional significance of these time locked 
phasic changes in facilitation and inhibition. We argue that 
facilitation is achieved by the alignment of a central 
attentional system with the pathway indicated by the cued event. 
This view is implied by the finding that both central and 
peripheral cues produce facilitation. Inhibition depends more 
upon the sensory information presented in the cue since there is 
no inhibition following the central cue. 

According to this functional hypotheses, facilitation and 
inhibition work together in the process of visual orienting in 
the following way. When the eyes are fixed, a peripheral visual 
stimulus tends to summon attention rapidly to its location. 
Attention marks this area and gives priority to information 
there. Thus, cued locations are processed faster. Under some 
conditions a movement of the eyes will follow in the direction of 
the facilitated area. As the eyes move, attention is reoriented 
back to the fovea. It is not necessary for an eye movement to 
induce attentional orienting to the fovea since there are 




conditions in which facilitation moves in retinotopic coordinates 
(Posner & Cohen, 1984). However, in most situations the fovea is 
favored because objects of interest tend to be foveated and also 
because it serves to keep attention and the fovea coordinated 
during successive changes of eye position. When one reorients 
away from the target by an eye movement, the previously 
facilitated target location is inhibited so there is a bias 
against returning the eyes to the previously cued environmental 
location. While inhibition occurs with and without eye movements 
(Posner & Cohen, 1984) it appears closely related in a functional 
sense to the eye movement system. Thus, the inhibition effect 
would serve as one of many neural systems designed to favor 
novelty over repetition. In accord with this hypothesis, it 
has been found that inhibited positions are less likely to 
draw the eyes back to them (Posner, Choate & Vaughan, in 
preparation; Vaughan, 1984). 

While this hypothesis remains speculative, it does suggest 
that there are functional advantages for the component 
facilitations and inhibitions that have been found in experiments 
on spatial orienting. Thus they tend to link the component 
facilitations and inhibition to important ecological factors in 
the subject's visual environment. 

Neural Systems 

I hope the foregoing indicates that the study of covert 
orienting has provided a reasonably simple but rich functional 
model of an internal cognitive operation. Moreover, the ability 
to describe the operations of covert orienting in terms of time 





locked component facilitations and inhibition suggests that it 
can be related to underlying neural systems as well. For a 
hundred years in clinical neurology it is known that lesions, 
particularly of the right parietal lobe produced problems that 
can be described, at least in part, as a difficulty in 
orienting attention to the side of space contralateral to the 
lesion (see DeRenz i , 1982 for a review). These results from 
clinical neurology of course fit with the material that we have 
mentioned from EEG studies showing a reduction in the N 100 when 
patients with parietal lesions had their attention directed 
to one ear (Naatanan, 1982) and for single cell recording 
results suggestion that cells in the posterior parietal lote shew 
selective enhancement (Wurtz, et al , 1980 ). 

Ue have been studying such patients using experiments like 
those illustrated in Figure 2 (Posner, Cohen & Rafal, 1982; 
Posner, Walker, Friedrich £ Rafal, in press). Figure 4 shows 
the data of one typical parietal patient. Recently we have 
summarized similar data from thirteen s- ch patients (Posner, et 
a 1 , in press). This particular patient, R.S. had a 

Insert Figure 4 about here 

right parietal tumor which was excised, removing most of the 
right parietal lobe. In this study, 80S of the targets were on 
the cued side. The target remained present for only one second 
and we waited five seconds for a response. The patient serves as 
his own control because we contrast responses to targets on the 
contralateral side which go directly to the lesioned hemisphere, 








with targets ipsilateral to the lesion which go directly to the 
intact hemisphere. We reasoned that if parietal lesions 
interfere with covert orienting of attention, we ought to 
find great difficulty in the subject orienting to targets 
opposite the lesion. 

For trials where the target is on the cued side (solid 
lines) this patient showed a small but consistent advantage to 
targets ipsilateral to the lesion over those appearing 
contra1 atera1 to the lesion. In fact, some of the parietal 
subjects that we have run show no advantage on cued trials 
between the two sides. The time between the cue and the target 
did not affect this relationship strongly, suggesting that the 
results were not mediated by eye movements and also suggesting 
that the effect of the cue was similar irrespective of the side 
on which the cue occurred. 

We can next examine the relation between ipsilateral 
targets on the cued and uncued sides. These are shewn by the 
triangles. There is a clear cuing effect that emerges by 550 
millisec after the cue. This is a relatively slow cueing effect 
compared with our normal subjects. However, the advantage for 
the cued side is of a size that approximates normal. We have 
founa parietal patients who show both rapid ana slow cuing 
effects. 

The results are strikingly different when targets are 
presented on the side contra latera1 to the damaged hemisphere. 

If attention is drawn to the good side by a cue (open circles) 
there is massive inteference with the processing efficiency of 
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trie target. At short intervals this effect is so great in 
subject R.S. that the patient usually does not detect the target 
within the five seconds allowed. This represents a confirmation 
of fincings in clinical neurology where on double stimulation the 
subject is unable to report information contra1 atera1 to the 
lesion. however, even with intervals of 1000 millisec so that 
the cue is gone from the field there is massive interference with 
reaction time. 

In further work (Posner et al, in press) we have found 
the same massive interference when we brighten the central box 
thus drawing attention to the fixation location. The central 
cue provides no information on target location and yet it 
produces a massive interference with reaction time. This 
suggests subjects have difficulty with the mental operation which 
we call disengage. That is, they seen to be able to move their 
attention normally once it is disengaged but they have difficulty 
in disengaging to any event that occurs in a direction 
contralateral to the lesion. 

Our results with parietal patients contrast rather strongly 
with lesions of the mid-bra in obtained from patients suffering 
from progressive supranuclear palsy (Posner, Cohen & Rafal, 

1982). These patients show a progressive loss in the ability to 
move their eyes in vertical directions while often maintaining 
for some period of time the ability to move their eyes in the 
horizontal direction. If we compare covert orienting to targets 
in a vertical direction with those obtained with targets in a 
horizontal direction wo find that while subjects are able to 
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orient covertly in either direction when targets occur in a 
vertical direction the orientation is slowed. This slowing 
of orienting on both cued and uncued trials suggests that lesions 
of the mid-brain areas produce inteference not with the aisengage 
but with the move operation. 

Conclusions 

I believe that the studies of covert orienting using 
chronometric paradigms with normal and brain injured subjects, 
event related potential methods, and single cell recording 
provide considerable evidence in support of the view point 
outlined in our general framework. While the franework is 
rather general in orientation it does suggest some differences 
from views commonly asserted concerning the relationship 
between cognition and brain processes. 

First, some psychologists (eg Neisser, 1976) have rejected 
the study of elementary mental operations by chronometric 
techniques because they do not seem neither ecologically valid 
nor linked to the underlying biology. It seems to me that 
the general framework outlined in Figure 2 provides deep and 
detailed links between these experimental studies and the 
underlying biology of neural systems. 

Second, other critics have suggested that the event related 
potentials provide no meaningful links to the level of analysis 
at which individual neural cells perform information processing 
operations. However, the degree of convergence between EE6 
methods and single cell methods within our model spatial task 
seem to suggest that both are giving meaningful measures 



of underlying mental operations in the paradigms that we have 
studied. 


Third, there is no apriori way to ensure that the elementary 
mental operations postulated by computational models at the 
cognitive science level have a relationship to the mental 
activities that people perform. However, chronometric studies 
showing rather beautiful time locked re 1 ationships between 
component facilitations and inhiitions and these mental 
operations seem to suggest that in fact such theories often do 
postulate mental operations that are natural for human subjects 
to perform and provide meaningful analysis in terms of 
underlying components. 

Thus the evidence seems to provide significant links between 
each level outlined in the general framework. Such links are 
particularly useful in being able to provide the family 
of a patient suffering from a brain insult with information 
concerning cognitive deficits that are most likely to result. 

We are a long way from having a theory sufficiently deep to do 
this but it appears that the kinds of links that we have been 
able to achieve do provide the possibility for more detailed 
clinical application. 

In this paper I have attempted to provide something of an 
overview of modern cognitive science in relationship to the 
question of how cognitive processes are instantiated in neural 
systems. It is clear that this is a complex and largely 
unanswered question. However, I hope that I have provided some 
guide to the growing literature that suggests a possibility that 
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Figure Captions 

Fig. 1; A general framework for the linking of cognitive tasks 


to underlying neural systems. The top two rows represent 
active areas of cognitive science (dividing complex tasks 
into elementary operations), the third row indicates 
experimental psychology of the type described in this 
paper. The bottom rows deal with underlying neural 
populations and single cell activity. 

"ig 2: Results of an experiment on covert orienting with 12 

normal subjects. The solid figures (triangles, circles, 
squares) are for trials with targets on the cued side; 
the open figures on the uncued side. The dot-oash lines 
are results from day one where targets are equally likely 
to occur on the cued and uncued side. After day 1 six 
subjects were run in a condition where the targets 
occurred on the cued side .8 of the time and uncued .2 
(solid lines). The other six were * u n in a condition 
where the targets occurred on the cued side .2 and 
the uncued side .8 (dash lines). After Posner, Cohen & 
Rafal, 1982. (see text). 

Fig. 3: Three putative elementary operations involved in covert 
orienting task. 

Fig. 4; Results of one illustrative right parietal patient in the 
covert orienting task. The basic re suits for target in 
the right visual field (triangles) resemble normals, in 
the left visual field (circles), however, there is a 
profound disruption with the efficiency of orienting to 
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an uncued target. Cued targets are indicated 
lines and uncued by dash lines. After Posner, 
Rafal, 1982. (see text). 
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